DSA4262 Individual Assignment 2 (Predictive Modelling)¶

Tan Teck Hwe Damaen (A0259079M)¶

Introduction To The Project¶

Introducing Stress:¶

Stress has become an increasingly pervasive issue worldwide, affecting both mental and physical health. Studies indicate that a significant portion of the population experiences high stress levels regularly, which can contribute to conditions such as anxiety, depression and cardiovascular problems. With the rapid rise of social media, many users now openly discuss stressful incidents online, providing a rich source of data for understanding patterns of stress. Among various platforms, Reddit has emerged as a particularly informative space, where communities openly share personal experiences and challenges. The Dreaddit project leverages this growing body of user-generated content to study stress expressions on social media, aiming to systematically identify posts that reflect stressful experiences.

About The Dreaddit Project:¶

The Dreaddit project focuses on determining whether a given Reddit post can be classified as stressful or non-stressful. To achieve this, we incorporate multiple features derived from each post. These include raw text content, numerical metrics such as LIWC scores (capturing linguistic and psychological patterns), sentiment scores, confidence levels and DAL features. By combining these textual and numerical indicators, the project seeks to build a robust framework for accurately predicting the stress level of a post.

What The Project Is About:¶

To classify texts as stressful or non-stressful, we will develop a machine learning pipeline and evaluate its performance using F1-scores as the primary metric. The analysis will explore a wide range of models, including classical approaches like Logistic Regression and Support Vector Machines (SVM), ensemble methods such as Boosting models, as well as deep learning architectures including Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks. This diverse set of models allows us to compare performance across different techniques and identify the most effective approach for stress detection.

Datasets That We Will Be Using:¶

The datasets used in this study include dreaddit_train, which serves both training and validation purposes and dreaddit_test, which is reserved for evaluating the final model. The model achieving the highest F1-score on the validation set will likely be applied to the test dataset, and a feasibility analysis will be performed to ensure reliable performance in practice. This approach allows us to measure the generalizability of the model while maintaining rigorous evaluation standards.

Link To GitHub Repository:¶

Link: https://github.com/tadamaen/DSA4262/tree/main/DSA4262%20Individual%20Assignment%202

---

Importing Relevant Modules¶

1. Pandas - For Data Cleaning And Transformation
2. Numpy - For Data Manipulation
3. Matplotlib - For Data Visualization
4. scikit-learn - For Machine Learning
5. Seaborn - For Data Visualization
6. PyTorch - For Deep Learning

Note: I am using a different version of numpy. This is because torch only supports numpy 1.x versions and not numpy 2.x versions. Hence, I am using a numpy 1.x version instead. To run the code properly without any errors, please press "Restart Kernel And Run All Cells". Please ignore the error message that appears after the first block is run, the code works fine :)

Importing The Given Datasets¶

There are two datasets that will used in this Predictive Modelling project. These include:

1. dreaddit-train.csv
2. dreaddit-test.csv

NOTE: These two files are in csv format and we will import the csv files directly using pandas pd.read_csv method.

Importing Dataset 1 - Dreaddit Train (CSV File)¶

Importing Dataset 2 - Dreaddit Test (CSV File)¶

---

Data Cleaning¶

We will need to perform data cleaning for both the training and test sets. The data cleaning methods will include:

1. Removing unnecessary and irrelevant columns
2. Renaming columns to a more readable format
3. Standardizing the sentence range values (like removing inconsistencies [] vs (), number range)
4. Identification and handling of any missing values

Data Cleaning Process (Train + Test Sets)¶

Step 1: Removing Unnecessary Columns¶

Before proceeding with the visualizations, it is important to note that the Dreaddit training and test datasets contain a large number of features (116 columns). Many of these variables are not directly relevant to our analysis. Including an excessive number of features in subsequent predictive modelling may increase model complexity, raise the risk of overfitting and ultimately lead to poorer generalisation performance on the test set. Therefore, we perform a feature selection step to retain only the columns that are most relevant for exploratory analysis and model development.

A question that we should ask is which columns are considered important and which columns should we remove in order to reduce the number of columns as much as possible but still retain most of the important data for consideration? This is when we should explore each column one-by-one and determine whether the column is important.

Important Columns Include:

Category 1: Core Metadata And Annotation Columns

Column Name In Dataset	Revised Name	Why Important?
subreddit	Subcategory	Identifies the community context in which a post is written, enabling analysis of how stress expression varies across different social environments. It is essential for subgroup performance analysis and domain-specific interpretation
sentence_range	Sentence Range	Provides a coarse measure of post length and verbosity, which can reflect cognitive load or narrative stress. It is useful for analysing how stress correlates with the amount of text produced
text	Text	Contains the raw linguistic content of each post, which is necessary for qualittive inspection, error analysis, and feature extraction. It enables direct interpretation of model predictions and failure cases
label	Label	Represents the ground-truth stress annotation and serves as the target variable for supervised learning. It is required for both model training and performance evaluation
confidence	Confidence Level	Indicates the annotators confidence in the assigned label, reflecting annotation reliability. This feature is useful for analysing ambiguous cases and understanding disagreement in stress perception

Category 2: Sentiment And Social Context Features

Column Name In Dataset	Revised Name	Why Important?
sentiment	Sentiment	Captures the overall emotional polarity of a post, providing a high-level affective signal. It allows comparison between emotional tone and stress labels, highlighting cases where stress is expressed without overt negativity
social_upvote_ratio	Upvote Proportion	Measures the proportion of positive community feedback received by a post, serving as a proxy for social approval. It helps analyse how stressed versus non-stressed posts are received by the community
social_num_comments	Number Of Comments	Reflects the level of engagement a post generates, indicating social attention or concern. This feature provides contextual insight into how stress-related content elicits discussion

Category 3: LIWC Affective Features

Column Name In Dataset	Revised Name	Why Important?
lex_liwc_negemo	Negative Emotional Language	Measures the proportion of negative emotional language in a post. It is a strong and interpretable indicator of affective distress commonly associated with stress
lex_liwc_anx	Anxiety	Captures anxiety-related language, which is closely linked to psychological stress. This feature is particularly relevant for detecting anticipatory worry and tension
lex_liwc_sad	Sadness	Represents expressions of sadness, a frequent component of distress-related narratives. It helps distinguish emotionally heavy content from neutral discourse
lex_liwc_anger	Anger	Measures expressions of frustration and anger, which often co-occur with stress. It is useful for identifying stress manifested through irritability rather than sadness

Category 4: LIWC Cognitive Processing Features

Column Name In Dataset	Revised Name	Why Important?
lex_liwc_cogproc	Cognitive Processing	Captures cognitive processing language associated with thinking, reasoning, and mental effort. Stress is often linked to rumination and overthinking, making this feature highly informative
lex_liwc_insight	Introspective Language	Reflects introspective and self-reflective language. It is useful for identifying posts involving personal reflection, which often accompany stress narratives
lex_liwc_cause	Casual Reasoning Language	Measures causal reasoning language, indicating attempts to explain or rationalise situations. Such reasoning is common in stressed individuals trying to make sense of difficulties
lex_liwc_tentat	Tentative Language	Measures tentative language, signalling uncertainty and lack of confidence. Stress frequently manifests as hesitation or indecision in language use
lex_liwc_certain	Certainty	Represents expressions of certainty and conviction. It provides a useful contrast to tentative language, helping distinguish confident statements from uncertain, stress-related ones

Category 5: LIWC Self-Focus

Column Name In Dataset	Revised Name	Why Important?
lex_liwc_i	First Person Singular Pronouns	Captures personal pronoun usage more broadly. It reflects how much a post is centred on personal experience versus external topics
lex_liwc_we	First Person Plural Pronouns	This indicates collective focus or group orientation. High values suggest the writer is including themselves as part of a group.
lex_liwc_you	Second Person Pronouns	This indicates focus on the reader or another person. High values suggest the text is addressing someone directly.
lex_liwc_shehe	Third Person Singular Pronouns	This indicates focus on other individuals. High values suggest discussing someone else's perspective or actions.
lex_liwc_they	Third Person Plural Pronouns	This indicates focus on other groups. Useful for analyzing discussions about groups or communities.

Category 6: LIWC Social And Situational Stressors

Column Name In Dataset	Revised Name	Why Important?
lex_liwc_social	Social Interactions	Captures references to social interactions and relationships. Many stressors are interpersonal, making this feature valuable for contextual interpretation
lex_liwc_family	Family	Measures family-related language, which often reflects emotionally salient stress sources. It is useful for identifying relational and caregiving stress
lex_liwc_friend	Friends	Captures references to friendships and peer relationships. This feature supports analysis of social support and conflict in stress narratives
lex_liwc_work	Work	Measures work-related language, a common source of chronic stress. It is particularly relevant for subreddits focused on employment or career issues.
lex_liwc_money	Money	Captures financial language, reflecting economic pressure and insecurity. Financial stress is a major real-world stressor, making this feature highly interpretable.
lex_liwc_achieve	Achievement	Measures achievement-oriented language related to goals and performance. It helps identify stress arising from pressure to succeed or meet expectation
lex_liwc_risk	Risk	Captures language associated with danger and uncertainty. This feature is useful for identifying stress linked to perceived threats or instability

Category 7: LIWC Psychological Style And Social Expression

Column Name In Dataset	Revised Name	Why Important?
lex_liwc_Clout	Social Confidence	Captures degree of confidence, authority, or social dominance reflected in the language.
lex_liwc_Authentic	Authentic	Captures the extent to which language appears personal, honest, and self-revealing.
lex_liwc_Tone	Tone	Captures the overall emotional tone of the text (positive and negative emotional framing).

Category 8: DAL Semantic-Emotional Content

Column Name In Dataset	Revised Name	Why Important?
lex_dal_avg_activation	Emotional Activation	Captures the average level of emotional activation (calm to excited, passive to intense) expressed by words in the text.
lex_dal_avg_imagery	Mental Imagery	Captures the average vividness or concreteness of words (abstract thoughts vs. sensory, image-evoking language)
lex_dal_avg_pleasantness	Emotional Valence	Captures the average emotional positivity or negativity of words used

The other columns not listed in these categories are classified as not important or irrelevant for our visualization and modelling project, hence we will remove the other columns. We now have only 35 columns instead of the initial 116 columns, while still keeping most of the important information that can be used for our predictive models.

Step 2: Renaming Columns To A More Readable Format¶

Many of the columns names are still in unreadable formats like having underscores and names that do not make much sense. To keep all columns consistent, we will remove underscores and name all columns in camel case format.

Step 3: Standardizing Sentence Range Column Values¶

We observe that the sentence range column values are not standardized and exist in different formats. Some ranges are enclosed in round brackets () while others are enclosed in square brackets []. Let's explore the distinct values of the Sentence Range column.

We noticed that there are many different categories of sentence ranges. This might make it a problem for visualization as there will be too many categories for us to explore. We will standardize the sentence range column values to:

1. Very Short Text: 1 to 10 sentences
2. Short Text: 11 to 20 sentences
3. Medium Text: 21 to 30 sentences
4. Long Text: 30 to 50 sentences
5. Very Long Text: Over 50 sentences

To classify each text properly, we will take the two numbers and calcualate the mean of the two values. If the mean of the two values exists as a decimal, we will round the number up to the nearest integer. For example, if the sentence has a length range of (32, 37), we will first find the mean value, which is (32 + 37) / 2 = 34.5. The value of 34.5 is a decimal number and hence, we will round up the value to 35. Based on the classification above, a text of 35 sentences is under Long Text.

Step 4: Handling Missing Values

Next, we will examine the dataset to identify any missing values across the columns. This step is essential to ensure data integrity and completeness. If missing entries are detected, appropriate strategies such as row removal or data imputation will be applied to address them.

There are no missing values in both the dreaddit_train and dreaddit_test datasets, hence we do not need to apply any imputation and removal of any rows in the datasets. For now, the dreaddit training and test dataset has been cleaned and we can now proceed to exploratory data analysis.

NOTE: For the visualizations, we will only be using the training set and not the test set. (The only exception is the introductory visualization where we analyze the distribution of stressful vs non-stressful texts in order to check if the dataset is balanced or imbalanced)

---

Exploratory Data Analysis - Data Visualization¶

We now proceed with a series of data visualizations to further explore the dataset and uncover meaningful patterns. Each visualization is accompanied by analytical commentary that offers possible explanations for the observed trends and anomalies.

For the enlarged versions of the visualizations, please refer to this link: https://github.com/tadamaen/DSA4262/tree/main/DSA4262%20Individual%20Assignment%202/Exploratory%20Data%20Analysis

Introductory Visualization: Distribution Of Stressful VS Non-Stressful Texts In Training And Testing Sets¶

To better understand the composition of our dataset, we analyze the distribution of stressful versus non-stressful texts in both the training and testing sets. This comparison helps us identify whether the labels are balanced or skewed, which is critical for building reliable predictive models. If the data is highly imbalanced, the model may become biased toward the majority class, reducing its ability to correctly classify minority cases. In such situations, data rebalancing techniques such as undersampling, oversampling or synthetic data generation methods like SMOTE can be applied to improve model performance and ensure fair representation of both classes during training.

Dataset Distribution Insights:

Both the training and testing sets contain a marginally higher number of stressful texts compared to non-stressful ones.

• Train set: 1488 stressful vs. 1350 non-stressful.
• Test set: 369 stressful vs. 346 non-stressful.

The imbalance is present, though it is not severe. However, even small imbalances can influence predictive models, especially in classification tasks where minority classes may be underrepresented in learned decision boundaries. To ensure fairness and robustness, we will factor this imbalance into consideration during the predictive modeling stage.

---

Visualization 1: Distribution Of Positive And Negative Stress Labels By Sub-Category¶

This visualization examines the relationship between Subcategory (eg. PTSD, assistance, relationships) and Label (0 negative, 1 = positive indicator of stress). We first begin by computing the total number of texts within each subcategory to understand the overall distribution of data across categories. Next, texts within each subcategory are disaggregated by their labels - positive (label = 1) and negative (label = 0). To enable fairer comparisons across subcategories with differing sample sizes, the absolute counts are converted into percentages. This normalization ensures that subcategories with larger volumes of texts do not disproportionately influence the interpretation relative to smaller subcategories.

Visualization 1 Part 1: Total Number Of Texts Within Each Subcategory¶

Key Observations:

1. PTSD (584 texts), Relationships (552 texts) and Anxiety (503 texts) dominate the dataset. These three categories together account for nearly 60% of all texts in the training dataset, showing where most of the Reddit users concerns cluster.

2. Domestic Violence (316 texts), Assistance (289 texts) and Survivors Of Abuse (245 texts) form a second tier. These represent serious social and safety issues, but with fewer texts compared to mental health concerns.

3. Lower Frequency Categories include Homeless (168 texts), Almost Homeless (80 texts), Stress (64 texts) and Food Pantry (37 texts). They have much smaller number of texts and these categories highlight material needs and situational crisis. However, they are underrepresented in the dataset.

Possible Insights:

The dataset is heavily skewed toward psychological struggles (PTSD, Anxiety, Relationships) rather than material hardships (Homelessness, Food Pantry). This suggests Dreaddit users are more lilely to seek support for emotional and relational issues than for logistical or survival needs. The prominance of Relationships and Anxiety reflects the Draeddit community's role as a space for interpersonal and emotional support. Meanwhile, the smaller counts for Homelessness and Food Pantry suggest fewer users turn to Reddit for material aid compared to emotional support.

---

Visualization 1 Part 2: Number Of Positive And Negative Label Texts By Subcategory¶

The initial plot provides an overview of how texts are distributed across subcategories, but it does not convery any information about the nature or severity of stress expressed within those texts. A subcategory with a large number of posts may not necessarily indicate a higher prevalence of depressive indicators, it may simply reflect higher engagement or posting activity within that topic.

In the next visualization, the texts within each subcategory are further disagregated by their labels - positive (label = 1) and negative (label = 0) and the frequency of each label type is calculated. This deeper breakdown allows us to distinguish between subcategories that predominantly contain expressions of stress and those that do not, providing more meaningful insights into the emotional content of each subcategory rather than relying solely on volume-based comparisons.

Key Observations:

1. High Stress Categories - This includes Anxiety (331 stressful vs 172 non-stressful), PTSD (332 stressful vs 252 non-stressful) and Domestic Violence (202 stressful vs 114 non-stressul). These categories are strongly associated with emotional distress, aligning with clinical and social severity.

2. Low Stress Categories - Relationships (247 stressful vs 305 non-stressful) has more non-stressful texts, suggesting that not all relationship-related posts are framed as crises - many may be just seeking advice. Assistance (105 stressful vs 184 non-stressful) also leans toward non-stressful, indicating that requests for help may often be practical rather than emotionally charged.

3. Balanced Categories - Stress (35 stressful vs 29 non-stressful) and Survivors Of Abuse (112 stressful vs 133 non-stressful) are relatively balanced, showing that these topics can generate both stressful and non-stressful narratives depending on context.

---

Visualization 1 Part 3: Percentage Of Positive And Negative Label Texts By Subcategory¶

The previous visualization provides a useful breakdown of positive and negative labels within each subcategory, but using absolute counts alone can be misleading when comparing subcategories of vastly different sizes. For example, PTSD subcategory may have hundreds of texts while Food Pantry may only have a few dozen. Even if both subcategories have a similar proportion of positive labels, the raw counts would suggest that PTSD is overwhelmingly more "stressful" than Food Pantry, which is not necessarily true.

To address this imbalance, the next visualization is employed to convert the counts into percentages relative to the total number of texts in each subcategory. By using percentages, we can accurately interpret trends across subcategories regardless of their sample size, ensuring that similar subcategories are not overshadowed by larger ones.

Key Observations:

1. High-Stress Categories (Majority Stressful)

• Anxiety: 65.8% stressful, 34.2% not stressful
• Domestic Violence: 63.9% stressful, 36.1% not stressful
• Almost Homeless: 58.8% stressful, 41.2% not stressful
• PTSD: 56.8% stressful, 43.2% not stressful

2. Low-Stress Categories (Majority Not Stressful)

• Assistance: 63.7% non-stressful, 36.3% stressful
• Food Pantry: 62.2% non-stressful, 37.8% stressful
• Homeless: 62.5% non-stressful, 37.5% stressful

3. Balanced/Mixed Categories

• Relationships: 55.3% non-stressful, 44.7% stressful
• Survivors Of Abuse: 54.3% non-stressful vs 45.7% stressful (These sit near the middle, showing that experiences in these areas can be noth stressful and reflective/advisory)

Possible Insights:

Although in raw counts, PTSD and Anxiety looked dominant, but percentages show that Domestic Violence and Almost Homeless are equally or more skewed toward stressful texts. This highlights categories where distress is proportionally higher even if total volume is lower. Relationships is a complex category. Despite being one of the largest categories, relationships still tilts slightly towards non-stressful (55.3%), showing that not all interpersonal posts are crisis-driven, many may be just seeking advice or reflective ones.

---

Visualization 2: Effect Of Text Length On Stress Labels¶

To further understand how textual characteristics relate to stress indicators, we examine the relationship between text length categories (Very Short Text, Short Text, Medium Text, Long Text, Very Long Text) and the assigned labels (0 = Not Stressful Text, 1 = Stressful Text). First, we compute the absolute number of texts within each text length category that are labelled as positive and negative. This provides an initial sense of how label distributions very across different text lengths.

However, because some text length categories may contain substantially more texts than others, absolute counts alone may led to biased interpretation. Therefore, we will additionally compute the absolute percentages of positive and negative labels within each text length category. The percentage-based perspective allows for a fairer comparison across text lengths, highlighting whether longer or shorter texts are proportionally more likely to express stressful indicators.

Visualization 2 Part 1: Number Of Texts With Positive And Negative Labels By Text Length¶

To examine how stress labels vary across different text length categories, a heatmap is used to visualize the relationship between text length and label type. The heatmap displays the absolute number of texts within each text length category that are labeled as positive or negative. Color intensity represents magnitude, making it easy to identify patterns. A Viridis color scale is employed to ensure perceptual uniformity and accessibility.

Key Observations:

1. Very Short Text (1 to 10 sentences) dominate. There are 673 non-stressful vs 725 stressful texts, which represents tha largest category overall. This suggests that many users express distress or support in brief, concise posts rather than long narratives.

2. Short Text (11 to 20 sentences) are the second largest, with 268 non-stressful and 331 stressful texts, still skewed towards stressful. This indicates that short posts often carry emotional weight, possibly quick disclosures or urgent statements.

3. For the other categories like Medium Text (21 to 30 sentences), Long Text (31 to 50 sentences) and Very Long Text (Over 50 sentences), the number of non-stressful and stressful texts are more balanced.

Possible Insights:

1. Stress Is Expressed Concisely - The majority of stressful texts are short or very short showing that distress often comes in brief bursts of communication. This aligns with the idea that people in crisis may not elaborate but instead post quickly and urgently.

2. Longer Texts Are More Reflective In Nature - As text length increases, the balance between stressful and non-stressful posts evens out. Longer posts may include context, reflections or advice-seeking, diluting the overall stress signal.

---

Visualization 2 Part 2: Percentage Of Texts With Positive And Negative Labels By Text Length¶

The percentage-based heatmap illustrates the proportion of texts within each text length category that are labeled as stress or not stress texts, providing insight into how the likelihood of expressing stress-related content varies with text length. By examining percentages rather than raw counts, the visualization reveals patterns that are not driven by the volume of texts in each category but by their relative composition. For instance, even if very short texts appear frequently in the dataset, the heatmap allows us to assess whether they are proportionally more or less likely to indicate stress compared to longer texts.

Key Observations:

1. Short Text (10 to 20 sentences) show the highest proportion of stressful posts (55.3%) while Very Short Text (1 to 10 sentences) and Medium Text (21 to 30 sentences) also skew slightly towards stressful at 51.9% and 52.9% respectively.

2. Long Text (31 to 50 sentences) remain fairly balanced, with 51.7% stressful and 48.3% non-stressful. Interestingly, Very Long Text (Over 50 sentences) is the only category where non-stressful posts slightly outweigh stressful ones, at 51.1% compared to 48.9%. This suggests that while stress is more commonly expressed in shorter posts, longer narratives tend to balance out or even shift toward non-stressful framing.

Possible Insights:

Distress is often communicated in brief, urgent statements rather than lengthy explanations. Short texts are more likely to carry stressful content, which aligns with the ideas that indiciduals in crisis may not elaborate but instead post quickly and directly. Longer texts appear to dilute stress signals as they often include context, reflection and advice-seeking, causing more non-stressful framing to be introduced to the text.

A possible reason why Very Short Text category (1 to 10 sentences) have lower than expected proportion of stressful text is that extremely short messages often lack the emotional detail needed to be labelled as stressful. Many of these posts may be neutral stetements, clarifications or simple requests that do not explicitly convey distress. In contrast, Short Text (11 to 20 sentences) provide just enough context or emotional language to signal distress more clearly.

---

Visualization 3: Identifying Most Common Words Used In Stressful And Non Stressful Texts¶

To better understand the linguistic differences between stressful and non-stressful texts, we analyze the most frequently used words associated with each label. For each label, all texts are first combined into a single corpus, allowing us to capture overall word usage patterns within each category. Common words that do not contribute meaningful semantic information such as articles and conjunctions are then removed using a predefined list of stopwords. This preprocessing step ensures that the resulting visualizations highlight content-bearing words that better reflect emotional tone and thematic emphasis.

Two complementary visualizations are used. First, a word cloud provides an intuitive, high-level overview of commonly used words, where word size corresponds to frequency. Second, we plot the top 10 most frequently used words for each label using bar charts, where the x-axis represents words and the y-axis represents their frequency. Together, these visualizations allow us to identify both prominent themes and measurable differences in language usage between stressful and non-stressful texts.

Visualization 3 Part 1: Word Clouds Of Words Used In Stressful And Not Stressful Texts¶

The most prominent words in the word cloud include time, know, like, back, years, people, help, said, really, want, good, friends, life, feel, going, one and take. These words are larger, indicating higher frequency in non-stressful texts. Many of them are neutral or positive in tone - they suggest supportive or reflective contexts. Other words point to temporal framing, where users are recounting experiences or reflecting on the past.

The vocabulary is dominated by general conversational words rather than highly emotional or crisis-driven terms. Most words reflect everyday language patterns, suggesting that non-stressful texts often resemble casual conversations or reflective storytelling rather than urgent disclosures.

In the stressful texts word cloud, the most prominent words include like, dont, know, feel, get, time, even, want, anxiety, really, back, still, help, people, life, work and things. Compared to the non-stressful word cloud, there is a noticeable shift in tone and vocabulary. Stressful texts feature more negative or uncertain expressions which directly convey emotional struggle. Several words also suggest practical burdens or frustrations.

Possible Insights:

1. The prevalence of words such as friends, help and good in non-stressful texts suggests that non-stressful texts often revolve around social support, positive experiences or advice-seeking. These posts may be more about sharing life updates, seeking guidance or expressing gratitude rather than conveying distress. The frequent use of temporal markers like time, years and back indicates that non-stressful texts often involve reflection, where users recount past events in a calmer, less emotionally charged way.

2. Another insight is that while both stressful and non-stressful texts use common conversational words, the co-text and framing differ. In stressful texts, these words are embedded in narratives of struggle whereas in non-stressful texts, they appear in more neutral or positive exchanges. This highlights how linguistic overlap masks divergent emotional tones, making context crucial for classification.

Visualization 3 Part 2: Plotting 10 Most Frequently Used Words In Stressful And Non Stressful Texts¶

Key Observations:

The most frequent word is “like” (425), followed by “time” (349) and “get” (335). Other highly used words include “know” (308), “dont” (285), “really” (277), “one” (276), “people” (248), “years” (240), and “help” (230). The distribution shows that non-stressful texts rely heavily on general conversational words rather than emotionally charged or crisis-specific terms. Most of these words are common in everyday dialogue and reflective writing.

Possible Insight:

The vocabulary pattern indicates that non-stressful texts are conversational, reflective and socially oriented. Words like time, years and back point to temporal framing, where users recount experiences or reflect on the past. Meanwhile, people and help highlight community engagement and supportive interactions, showing that non-stressful texts often involve advice-seeking, offering help or discussing relationships in a positive or neutral tone.

Key Observations:

The most frequent word is “like” (756), followed by “dont” (649) and “feel” (575). Other highly used words include “know” (548), “get” (517), “even” (426), “time” (408), “really” (373), “cant” (346), and “want” (338). Compared to the non-stressful texts chart, stressful texts show a much higher frequency overall - for example, like appears almost twice as often in stressful texts (756 vs 425). The vocabulary also shifts toward emotionally charged and negative expressions. These words directly convey distress, uncertainty or limitation.

Possible Insight:

Stressful texts are characterized by higher intensity and repetition of emotionally loaded words. The prominence of dont and cant highlights themes of restriction, helplessness or barriers, while feel and want emphasize emotional states and unmet needs. The frequent use of even and still suggests ongoing struggles or frustration, reinforcing the sense of persistence in distress.

---

Visualization 4: Exploring Social Upvote Ratio Distributions On Stressful And Non-Stressful Texts¶

The social upvote ratio is a metric that reflects the relative approval of a post or text by the online community. It is calculated as the proportion of upvotes to the total number of votes (upvotes + downvotes) that a post receives. A higher social upvote ratio indicates that a larger proportion of users found the post helpful, relatable or valuable, whereas a lower ratio may suggest disagreement, irrelevance or negative reception.

Stressful texts, which often describe personal struggles, distress or sensitive experiences, may elicit more empathetic responses from readers, potentially resulting in higher upvote ratios if the content resonates with the community. Conversely, non-stressful texts, which may be neutral, informational or casual, might not engage the same emotional response and their social upvote ratios could vary differently.

A boxplot is employed, with one boxplot for stressful texts and another for non-stressful texts. Boxplots provide a concise summary of the distribution, highlighting key statistics such as the median, interquartile range, minimum and maximum values. By comparing the boxplots, we can visually assess differences in central tendency, spread and potential outliers between stressful and non-stressful texts. The primary aim is to determine whether stressful posts consistently receive higher community engagement as measured by the social upvote ratio.

Key Observations:

1. Both stressful and non-stressful texts reach the same maximum upvote ratio of 1.0, meaning that some posts in each category were universally approved by the community. However, the median upvote ratio is slightly higher for stressful texts (0.91) compared to non-stressful texts (0.87). Similarly, the first quartile (Q1) is higher for stressful texts (0.76) than for non-stressful texts (0.72), suggesting that even the lower-performing stressful posts tend to receive stronger approval than their non-stressful counterparts.

2. The minimum values show a small difference: stressful texts dip as low as 0.14 while non-stressful texts bottom out at 0.17. This indicates that while both categories can receive poor reception, stressful texts have a slightly wider spread in community response. Overall, the distribution suggests that stressful texts are more consistently well-received with higher central tendency values.

Possible Insights:

The data supports the idea that stressful texts elicit more empathetic and supportive responses from the community. Posts describing personal struggles or distress may resonate more deeply, prompting users to upvote as a gesture of solidarity or encouragement. This explains why stressful texts show higher median and quartile values in their upvote ratios. Non-stressful texts may be more neutral, informational or casual. While they can still achieve high approval, they do not consistently generate the same empathetic engagement. Their slightly lower median and quartile values suggest that readers may be less compelled to upvote unless the content is particularly useful or relatable.

---

Visualization 5: Exploring Number Of Comments On Stressful And Non-Stressful Texts¶

The number of comments associated with a post provides an additional social engagement signal that may help distinguish between stressful and non-stressful texts. Posts that express distress, anxiety or emotional vulnerability are more likely to elicit responses from the community such as offering advice, emotional support or shared experiences. In contrast, non-stressful texts may attract fewer comments or more neutral interactions.

To examine this relationship, we visualize the distribution of the number of comments for stressful and non-stressful texts using a violin plot. Compared to a boxplot, a violin plot captures not only summary statistics (such as the median and interquartile range) but also the full distributional shape, making it particularly suitable for identifying skewness, multimodality and extreme values. This visualization allows us to assess whether stressful texts tend to generate a higher volume of comments or exhibit greater variability in engagement, thereby offering deeper insight into how users respond to different types of content.

Visualization 5 Part 1: Distribution Of Number Of Comments By Text Type (Absolute Number)¶

The violin plot generated highlights a key limitation: the raw distribution of comment counts is extremely skewed. There are several texts in both stressful and non-stressful texts that have many comments which make the maximum have much higher values. This makes the visualization unsuitable for deeper analysis because it hides subtle differences in spread and central tendency.

A better next step is to experiment with alternative transformations such as the square root transform. Unlike the log, the square root reduces skew while still preserving more variation in the mid-range values. This helps us to see differences in quartiles and medians without letting extreme outliers dominate the scale. By doing so, the interpretability of the raw counts will be retained while making the distribution shape clearer and the summary statistics more meaningful for comparison.

Visualization 5 Part 2: Distribution Of Number Of Comments By Text Type (Square Root Transforme)¶

Key Observations:

The square-root scaled violin plots show that both stressful and non-stressful texts share nearly identical quartiles: Q1 ≈ 1.41, median ≈ 2.24, and Q3 ≈ 3.16, with a minimum of 0. This indicates that the bulk of the data is concentrated at low comment counts, regardless of text type. The maximum values differ slightly, with stressful texts reaching ≈ 17.8 and non-stressful texts ≈ 20.4. The violins themselves are narrow at the base, widen around the median and taper toward the maximum, reflecting a skewed distribution with many low-comment texts and a few extreme outliers.

Possible Insights:

The identical quartiles likely result from two factors: the square-root transformation, which reduces skew but can map multiple discrete values to the same transformed positions and the inherently discrete nature of comment counts. The higher maximum for non-stressful texts contradicts the hypothesis that stressful texts attract more engagement. This discrepancy may be due to outliers: a few non-stressful posts could have gone viral or received unusually high attention because of topic popularity, posting time or community dynamics. Since maximum values are highly sensitive to single extreme cases, they are not reliable indicators of overall engagement.

---

Visualization 6: Plotting The Distribution Of Sentiment Values From Stressful And Non Stressful Text¶

This visualization aims to examine the distribution of sentiment values in stressful and non-stressful texts, with particular emphasis on identifying the modal sentiment ranges for each group. Sentiment scores provide a numerical summary of emotional tone, ranging from negative to positive values. By observing where sentiment values concentrate most frequently, we can assess whether stressful texts tend to cluster around more negative sentiment ranges while non-stressful texts may center around neutral or slightly positive values. Understanding these differences is important because sentiment polarity often reflects underlying emotional distress, frustration or anxiety which are key indicators of stress.

A histogram is used for each text category with a fixed bin width of 0.1, spanning the full sentiment range from -1.0 to +1.0. Using a constant bin interval ensures fair and consistent comparison between stressful and non-stressful texts, allowing differences in modal ranges and distribution shapes to be clearly observed. Histograms are particularly suitable here because they reveal both the frequency and spread of sentiment values, making it easy to detect skewness, clustering and overlaps between the two groups.

Key Observations:

The histogram of sentiment values for stressful texts shows a strong clustering around neutral to slightly negative sentiment. The highest frequencies occur at sentiment values of between -0.1 and 0.0 (344 texts), followed closely by between 0.0 and 0.1 (329 texts). This indicates that most stressful texts are not extremely polarized but instead hover near the neutral range. At the extremes, very few texts fall below -0.5 or above 0.5, with counts dropping sharply as sentiment moves toward -1.0 or +1.0.

Possible Insights:

The concentration of stressful texts around neutral sentiment suggests that stress does not necessarily manifest in overtly negative language. Instead, many stressful posts may be expressed in a balanced or matter of fact tone, which sentiment analysis interprets as neutral. The scarcity of extreme values implies that highly emotional or strongly positive/negative language is rare in this dataset. This could be due to the nature of the platform or community, where users describe stressful experiences in a restrained way rather than venting with extreme sentiment. The slight skew toward negative values aligns with expectations that stressful texts lean towards a more negative sentiment.

Key Observations:

The histogram of sentiment values for non-stressful texts shows a concentration around neutral to slightly positive sentiment. The highest frequencies occur between 0.1 and 0.2 (338 texts), followed closely by between 0.0 and 0.1 (310 texts), indicating that most non-stressful texts lean toward neutrality or mild positivity. Negative sentiment values are present but far less frequent, with counts dropping sharply below -0.2. The distribution is roughly bell-shaped, tapering off toward both extremes, but with a noticeable skew toward the positive side compared to stressful texts.

Possible Insights:

The clustering around neutral and slightly positive values suggests that non-stressful texts are often expressed in balanced or optimistic tones, which sentiment analysis captures as positive. This contrasts with stressful texts, which clustered around neutral to slightly negative values. The difference implies that stress influences language tone, but not in an extreme way. Stressful texts avoid strong negativity, while non-stressful texts lean gently positive. The higher peak at 0.1 for non-stressful texts may reflect community norms of supportive or casual communication, where neutral posts sometimes carry a slight positive framing.

---

Visualization 7: Exploring LIWC Linguistic Features and Their Relationships with Stressful vs. Non-Stressful Texts¶

The Linguistic Inquiry and Word Count (LIWC) tool is widely used in computational linguistics and psychological research to quantify the presence of specific psychological, emotional and cognitive processes in text. LIWC categorizes words into several meaningful psychological dimensions, such as affective states (anxiety, sadness, anger), cognitive processes (causation, insight, certainty), self-focus (use of first-person pronouns), and social or situational stressors (family, work-related terms).

In the context of analyzing stressful versus non-stressful texts, these LIWC factors may reveal meaningful patterns. For example, texts with higher anxiety or anger scores may be more likely to be labeled as stressful whereas texts with higher positive affect or cognitive processing words might correlate with non-stressful texts.

In this visualization, we focus on six main categories:

1. LIWC Affective Features - capturing emotions such as anxiety, sadness, anger and positive affect.
2. LIWC Cognitive Processing Features - including analytical thinking, causation, certainty and insight.
3. LIWC Self-Focus - measuring the degree to which texts use first-person singular or plural pronouns, reflecting self-orientation or self-involvement.
4. LIWC Social and Situational Stressors - representing references to family, friends, work or other social stress contexts.
5. LIWC Psychological Style and Social Expression Features - representing references to social confidence, authenticity and tone
6. DAL Semantic-Emotional Content Features - includes features such as emotional activation, mental imagery and emotional valence

For each of these categories, we will:

1. Use boxplots to visualize the distribution of LIWC scores across stressful and non-stressful texts, providing insight into how different psychological or linguistic factors vary between the two groups.
2. Plot correlations between each subcategory within the LIWC group and the label, allowing us to quantify and compare the strength of association between linguistic features and text stressfulness.

This combined approach provides both a descriptive and quantitative understanding of how psychological and linguistic markers manifest in stressful versus non-stressful communication, helping to highlight the most informative features for subsequent modeling or analysis.

Visualization 7 Part 1: LIWC Affective Features¶

First, we will create boxplots to visualize the distribution of LIWC scores across stressful and non-stressful texts for the LIWC affective features, which include: Negative Emotional Language, Anxity, Sadness and Anger.

Key Observations:

Negative Emotional Language stands out with the highest values overall, especially in stressful texts where the median and upper quartile are clearly elevated compared to other categories. Anxiety, Sadness, and Anger show much lower distributions, with many values clustering near zero for non-stressful texts. Sadness and Anger in particular reveal very compressed ranges in non-stressful contexts, suggesting that these emotions are far less frequently expressed compared to negative emotional language more broadly. Anxiety, while generally low, shows a wider spread in stressful texts, indicating that stress tends to amplify variability in anxious language use.

Differences between stressful and non-stressful texts:

Across all four categories, stressful texts consistently exhibit higher medians, quartiles and maximum values than non‑stressful texts. For example, stressful texts in Negative Emotional Language reach a median of nearly 4 compared to about 1.5 in non-stressful texts, while the upper quartile doubles from around 3 to over 6. This pattern suggests that stress not only increases the intensity of emotional language but also broadens its variability, making emotional expression more pronounced and diverse compared to non-stressful contexts.

Now, we will explore the correlations between each subcategory within the LIWC group of affective features and the label, allowing us to quantify and compare the strength of association between linguistic features and text stressfulness.

Key Observations:

Negative Emotional Language shows the strongest correlation with the stress label, around +0.39, which suggests that texts containing more negative emotional words are most predictive of being stressful. Anxiety follows with a moderate positive correlation (+0.26), while Anger (+0.22) and Sadness (+0.20) are slightly weaker but still positive. This ordering indicates that while all affective features contribute to distinguishing stressful texts, broad negative emotional language is the most dominant signal.

Possible Insights:

The fact that all correlations are positive means that higher values of these affective features consistently align with stressful texts, while lower values align with non-stressful texts. The relatively stronger correlation for Negative Emotional Language suggests that stressful texts are not only more likely to contain specific emotions like anxiety or sadness, but also a general increase in negative affective vocabulary overall. Meanwhile, the weaker but still positive correlations for Sadness and Anger imply that these emotions are present but less consistently predictive compared to anxiety or broad negativity.

---

Visualization 7 Part 2: LIWC Cognitive Processing Features¶

Next, we will create boxplots to visualize the distribution of LIWC scores across stressful and non-stressful texts for the LIWC Cognitive Processing features, which include: Cognitive Processing, Introspective Language, Casual Reasoning Language, Tentative Language and Certainty.

Key Observations:

Cognitive Processing shows the highest overall values among the features, with medians around 13-14 and upper quartiles reaching 16-18. This suggests that both stressful and non-stressful texts are rich in cognitive markers, reflecting reasoning and analytical language. Introspective Language and Tentative Language occupy a middle range, with medians around 2-3 and upper quartiles near 4-5, indicating moderate use of self-reflective and uncertain phrasing. Casual Reasoning and Certainty are lower overall, with medians closer to 1 and upper quartiles around 2-3, suggesting that explicit causal explanations and strong assertions are less frequent compared to broader cognitive processing or tentative expressions.

Differences between stressful and non-stressful texts:

Across most categories, stressful texts show slightly higher values than non-stressful texts. For example, Cognitive Processing in stressful texts has a higher median (13.99 vs. 12.77) and upper quartile (17.56 vs. 16.27), indicating more complex reasoning under stress. Certainty is slightly higher in stressful texts at the median but has a lower maximum, suggesting that while stressful texts may include more frequent assertions, they avoid extreme certainty compared to non-stressful texts. Tentative Language remains nearly identical between the two categories, showing that uncertainty is expressed at similar levels regardless of stress.

Now, we will explore the correlations between each subcategory within the LIWC group of cognitive features and the label.

Key Observations:

Casual Reasoning Language shows the strongest positive correlation (+0.11), suggesting that when people are stressed, they tend to use more causal explanations in their language. Cognitive Processing and Introspective Language both follow closely at +0.10, indicating that stressed texts are slightly more likely to contain analytical or self-reflective language. Certainty has a weaker positive correlation (+0.03), implying that while stressed texts may include more assertive statements, the effect is minimal. Tentative Language stands out as the only feature with a negative correlation (-0.02), suggesting that uncertainty is slightly less common in stressful texts compared to non-stressful ones.

Possible Insights:

The overall pattern shows that stressful texts lean toward more reasoning, reflection and cognitive engagement, albeit with modest effect sizes. The positive correlations for Cognitive Processing, Introspective Language and Casual Reasoning suggest that stress prompts individuals to think more deeply and explain more often, possibly as a way of coping or rationalizing their experiences. The slight negative correlation for Tentative Language indicates that stressed individuals may be less likely to hedge or express uncertainty, instead favoring more direct or explanatory language.

---

Visualization 7 Part 3: LIWC Self Focus Features¶

Next, we will create boxplots to visualize the distribution of LIWC scores across stressful (label = 1) and non-stressful (label = 0) texts for the LIWC Self Focus features, which include: First Person Singular Pronouns, First Person Plural Pronouns, Second Person Pronouns, Third Person Singular Pronouns and Third Person Plural Pronouns.